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Data Filtering 

The present invention relate, to the field of data filtering and, in partienlar i„ 
certam preferred embodiments to the field of deblocking filters in video decoders. ' 

In the field of video decoders, for example, in MPEG decoders, i, is known to 
use deblocking fiitets to improve the pictnre. Th.s is particularly important in 
stations where there is a low bit rate, as for example, in the transmission of v.deo 
data to a mobile telephone. 

Deblockmg filters consist of some decision logic, which is used to decide 
wnere tire boundary of a block is to be found, i.e. to detect a horizontal arrd/or vertical 
.me and then to decide whether the line is part of the picture or is merely the boundary 
of a block and as such should be removed. This decision logic is then Mowed by a 
low-pass filter that can be applied across the block boundaries. 

An example of a deblocking filter which is used in the MPEG-4 standard is 
given below. The matrix notation of this filter is shown in Figure 1. 

The impulse response of the filter is: 

(1,1,2,2,4, 2, 2, 1,1)/ 16 

Whan appiied the filter operates on an input array of ten pixels (xO to x9) and 
updates the middle eight pixe.s (y! ro y8). Whenever the filter would access pixels 
outs.de the input data, ,he va!ue of the appropriate boundary pixel is substituted A 

of a low pass filter that acts as a deblocking filter in an MPEG decoder. 

One drawback of applying this filter to the decoded data is that i, is expens.ve 
to do, and m some implementations requires twice the number of cycles as the MPEG 
4 decoder itself. 

A first aspect of the present invention provides a method of filtering data 
compnsmg: app.ying a plurafity of m fifter coefficients which each have a value which 
» a negative power of two or is zero and which sum to one, to a plurality of m i„ put 



data rtems to produce . filtered outpu, data item , by perform ,„ g , 

av^ng emulations, co m prisi„ g averaging mput data ° f 

— fii ,er coefficient „ «„ be appiied to pr oduoe first av e r a g ed data L Z ^ 

10 * , «. 1116 PreSeM inVeMi0n reCOgDiSeS Pr ° PertieS ° f «»- coefficients i e 
*at tbev ace g enera, Iy „e g afive powers of two and sun, ,o one and use h ^ 

data wrdth of an avera g e of two numbers is never ,ar g e r than the data ^ „ f ^ 

larger of the two numbers Thus if th~ ■ .a 

umoers. inus, it the input data were a series' of 8 k; + u 

significantly reduce the number nf ,w * 

me number of registers required to perform the flit. • 
caicuiations and thus can en.iv. , , perrorm the filtering 

~, tbe S e,uence of a.ra gmg ^^"CT 4 

av ra ged data and avera gmg the fct averaged da(a ^ avera „ ed data Qr 

30 2^r h ^^~ h ~-^-^ 
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In preferred embodiments, the sequence of averaging calculations further comprises 
successive further averagings, wherein averagings are performed between sets of input 
data, the sets of input data comprising averaged input data or individual input data 
t«ems, averages only being performed between sets of input data where the sum of the 
filter coefficients to be applied to each set of input data are equal. 

The sequence of averaging calculations are performed in a certatn way so that 
the filter coefficients are appropriately app.ied to the input data. The rules for this are 
set ou, above and examples of averagings that follow these ru.es are given below and 
nrvo.ve averages such as averaging a pair of averaged input data items to which a 
smaller coefficient is to be applied with an input data item to which a filter coefficient 
that is twice the value of the smaller coefficient is to be applied, or averaging two 
averaged pairs of input data items «o which the smaller coefficient is t0 be apobed 
etther wtth an input data item to which a filter coefficient that is four times the value of 
the smaller coefficient is to be applied, or with an averaged pair of input data items to 
which the filter coefficient mat is twice the va,ue of the smaller coefficient is to be 
applied. 



Although the fiber coefficients may consist of a single row of coeffictents to be 
■0 apphed to a plurality of input data items ,o produce a single output data hem in 
preferred embodiments h comprises repeatedly performing the steps of a first aspect of 
•he present invention such that an array of m r n filter coefficients are applied to an 
array of m input data items to produce a plurality of n output data items 



In preferred embodiments, the step of performing a sequence of averaging 
calcinations includes Meeting the input data that is to be averaged in dependence upon 
the frequency that two pieces of input data have common fiber coefficients applied to 
them, aard where posstble selecting to average two p.eces of data tha, have common 
filter coefficients apphed to them a plurality of times, such tha, the calculated average 
of the input data can be re-used. 



The way tha, the emulations are performed means .ha, the average of certain 
- ^ he use, in cations for different output da,a hem ^ 
W common filter coefficients a PP „ed ,o them. There is some choice in the da, 
terns tha, are to he averaged provided ,ha, certain rules are foUowed. Thus a way L 
^rove performance of rhe filter is to seiect da t a to he averted in such 1 ^ 

:r:ir - ges - — - — — - — and fc 



In some embodiments the filter coefficients have me required properties of 
.0 h negafive power of two and summing to one, whereas in others they may no 
Th, does no,, however, necessarily mean tha, embodimen«s of me invention canmo, he' 
»d ,n such _ances. In some embod.me n «s the method ^ ^ 

2 of. Poor ,o applying ,he p,ura,i,y of filter coe ffi cie„, s t„ the mpu, data, a^anging 

Preferably, the step of arranaing the fi i t(iT . ~- . 
~ . , n<uiging the filter coefficients such that ^ a -h 

oefficren, as a vaiue tha, is a negative power of two, compnses siting any fit 
efficen, ra, doea no, have mis property m,„ a aum of components, each om onem 
20 havmg a value that is a negative power of two. 

Thus, in the example given in the introduction for the MPEG-4 filter the 6s 
appearmg in me corners of the filter coefficients can he div.ded into a 4 + 2 and 

» r,r ,nput daa The - - - - — - — d; 

Although the method can he appfied to a plurality of differen( fi , ters 
Ptut-culariy appficahle to a situation where the fiher coefficients relate to a W 
filter, and preferably, a deblocking filter. P 
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data and can be used advantageously, in situations where the input data comprises an 
array often pixels, the filter coefficients being such that output data comprises an array 
of eight pixels. Thus, the filter acts on an input array often pixels, and updates the 
middle eight pixels. 

A further aspect of the present invention provides, a computer program product 
for applying a plurality of m filter coefficients to a plurality of m input data items 
comprising: filtering logic operable to control a processor to combine filter coefficients 
which each have a value that is a negative power of two and that sum to one, with a 
plurality of m input data items to produce a filtered output data item, by controlling the 
processor to perform a sequence of averaging calculations, the sequence of averaging 
calculations comprising averaging input data items to which a smallest filter 
coefficient is to be applied to produce first averaged data and averaging the first 
averaged data with other averaged input data or with input data items to which larger 
filter coefficients are to be applied the plurality of m filter coefficients being applied to 
the plurality of m input data items via a sequence of averaging calculations such that a 
data width of any calculated data does not exceed that of the input data items. 

Although the input data items can be of any size in some embodiments, they 
20 each comprise 8 bits. This makes them particularly adapted for handling by many 
conventional processors. 
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Although the filtering logic may comprise a number of different forms, in 
preferred embodiments it comprises a plurality of SIMD instructions. This form of 
25 instructions is particularly well adapted for handling these calculations. 

The use of SIMD instructions for handling 8-bit pieces of data means that in 
some processors having 32 bit registers four pieces of input data can be handled in 
parallel. This increases the speed of processing of the calculations considerably. 

30 

Preferably, the filtering logic is operable to set the carry flag to indicate if the 
filtered output data is to be stored or discarded. 



A property of embodiments of the present invention is that as the calculations 
performed are all averages, the carry flag is never needed, and it can therefore be used 
to indicate other things, such as whether data is to be stored or can be discarded. This 
can be particularly important in some embodiments, where the decision between 
filtering or not filtering is made for individual input data items that are being 
calculated in parallel and for which some results may need to be stored and some 
discarded. 



10 A still further aspect of the present invention provides A data processing 

apparatus comprising a processor operable to filter input data, the data processing 
apparatus comprising: filtering logic operable to control the processor to apply a 
plurality of m filter coefficients which each have a value that is a negative power of 
two and which sum to one, to a plurality of m input data items to produce a filtered 
15 output data item, by controlling the processor to perform a sequence of averaging 
calculations the sequence of averaging calculations comprising averaging input data 
items to which a smallest filter coefficient is to be applied to produce first averaged 
data and averaging the first averaged data with other averaged input data or with input 
data items to which larger filter coefficients are to be applied the plurality of m filter 
20 coefficients being applied to the plurality of m input data items via a sequence of 
averaging calculations, such that a data width of any calculated data does not exceed 
that of the input data. 



Particular embodiments of the present invention will now be described by waj 
25 of example only with reference to the accompanying drawings, of which: 

Figure 1 shows the matrix notation of a known low pass filter which acts as a 
deblocking filter in an MPEG decoder; 

Figure 2 schematically shows the blocks on a decoded MPEG-4 video picture 
30 prior to deblocking filtering; 
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Figures 3A and 3B show a filtered pixel calculated as a sequence of averages 
of the input pixels; and 

Figure 4 shows the application of filter coefficients that are not initially 
5 negative powers of two. 

Figure 2 schematically shows a video picture that is formed from MPEG-4 
encoded and then decoded data, prior to any deblocking. As can be seen it comprises 
blocks of data. A and B comprise lines of pixels which cross the block boundaries. It 
10 is these lines of pixels that need to be filtered by any deblocking filter to remove the 
"block" effect from the picture 

The deblocking filtering of any data occurs after the decoding. MPEG 
decoding involves feedback of some data and thus it is important that any calculations 
15 are accurate as errors can be magnified by the feedback process. The deblocking 
filtering occurs after the feedback loop and as such it is less important if the calculated 
data is not bit exact. 

A deblocking filter consists of some decision logic, which is not relevant here, 
20 followed by a low-pass filter that is applied across the block boundaries (A and B). 

It has been noticed that low pass filters used in deblocking generally have filter 
coefficients that are negative powers of two and that add to one. It was noticed that 
this property of the filter coefficients could be used to enable the output data to be 
25 calculated using a series of averaging operations. This would have the advantage that 
the working data is represented by a fixed number of bits throughout the calculation. 
It should be noted that calculating the output data in this way may not be completely 
bit exact as there will be some rounding of data at the various averaging stages. 
However, as noted above this is not important at this point in the process. 
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Using the example of the deblocking filter given in the introduction which 
operates on an input array often pixels and updates the middle eight pixels (see Figure 
1), y5 can be calculated as a sequence of averages as is shown in Figure 3 A and 3B. 

There are several possible ways of averaging the input data to caclulate y 5 
•Figure 3A shows one possible way in which y5 can be calculated as a sequence of 
averages, while Figure 3B shows another. These two possibilities are shown as a 
sequence of averaging operations using a tree representation. This procedure is very 
similar to building a Huffman tree. It should be noted at this point that the use of 
Huffman trees in regard to deblocking filters is not something that has been 
contemplated before, nor is it at all obvious to do so given the different field in which 
Huffman trees are generally used. For each row in the matrix an appropriate tree can 
be chosen. 

As can be seen from Figures 3A and 3B, there are several possible ways that y5 
can be calculated, each involving some difference in the input values that are being 
averaged. The possibility of averaging different input values and still arriving at the 
same answer enables one to choose different ways of performing the calculation This 
enables input values that have the same filter coefficients applied to them in several 
different calculations to be chosen to be averaged together, so that this averaging result 
can be re-used in a later calculations. For example, the averaging of x6 and x7 could 
also be used in the calculation of y 3 and yg. Similarly, x8 and x9 can also be used in 
the calculation of y6. 

Thus, for each row in the matrix, an appropriate tree is chosen. The optimal 
choice of trees provides the greatest number of common sub expressions between the 
rows in the matrix, subject to the number of available CPU registers. 

To aid in the formation of the trees, it is often possible to factorise the original 
filter into the product of filters with impulse response. The filter shown in Figure 1 
can be factorised into the product of two filters with impulse response (l,l,101)/4 



9 

i 



10 



and (1. 0, 1, 1, 1) / 4. This hints at the best tree composition but it can be improved 
upon. 

It should be noted that the averagings are performed according to a Huffman 
tree distribution and this results in averages only being performed between sets of 
input data where the number of averaged input data items times the filter coefficient 
that is to be applied to them are equal. That is to say x8, x9, xl and x2, each have 
filter coefficients of 1 applied to them so they are averaged first. If the average of all 
four is used, then these must be averaged either with a data input item having a filter 
coefficient of four applied to it (x5) or with the average of two data items having a 
filter coefficient of two applied to them (an average of two of x3, x4, x6 or x7) and so 
on. 



As can be seen each data point is generally used exactly once. However, in 
1 5 some cases, a filter coefficient may not be a negative power of two. See for example, 
the 6s in the corner of the matrix shown in Figure 1. These are dealt with by splitting 
them into a sum of 2 + 4. This splitting of the coefficients is generally done by the 
programmer and not by the processor, although the processor could in some 
embodiments be programmed to perform such an operation. Thus, the amended split 
20 coefficients are input into the processor. Figure 4 shows how this could be applied 
when calculating y8. 

In an embodiment of the present invention, this idea can be implemented using 
the ARMv6 (Advanced RISC Machines of Cambridge England) 8-bit SIMD 
25 instructions. This means that the deblocking filter can be implemented in 8-bit SIMD, 
allowing four input values to be calculated in parallel. This gives a factor of two 
improvement in speed over 16-bit SIMD. This is very attractive given the fact that 
generally the deblocking filter is twice as expensive as the MPEG-4 decoder is. 

30 A further advantage of embodiments of the present invention is that as the data 

width never increases throughout the calculation, the carry flags are never needed. 
These can therefore be used to carry other information. In a deblocking filter 



10 

according to an embodiment of the present invention the decision between filtering 
and not filtering is made at each boundary pixel. As the implementation processes 
four pixels in parallel, not all of them may be required. Before writing the set of four 
pixels back to the frame buffer, the SEL instruction selects the appropriate 
5 combination of updated and/or original pixels. These are indicated by information 
held in the carry flags. 



Although a particular embodiment of the invention has been described herein, 
it will be apparent that the invention is not limited thereto and that many modifications 
0 and additions may be made within the scope of the invention. For example, various 
combinations of the features of the independent claims could be made with the features 
of the dependent claims without departing from the scope of the present invention. 
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CLAIMS 

1 . A method of filtering data comprising: 

applying a plurality of m filter coefficients which each have a value which is a 
negative power of two or is zero and which sum to one, to a plurality of m input data 
items to produce a filtered output data item, by performing a sequence of averaging 
calculations, the sequence of average calculations comprising: 

averaging input data items to each of which a smallest filter coefficient is to be 
applied to produce first averaged data; and 

averaging the first averaged data with other averaged input data or with input 
data items to which, larger filter coefficients are to be applied, the plurality of m filter 
coefficients being applied to the plurality of m input data items via a sequence of 
averaging calculations such that a data width of any calculated data does not exceed 
that of the input data being averaged. 

2. A method according to claim 1, wherein the sequence of averaging calculations 
comprising averaging two items of input data to which a smallest filter coefficient is to 
be applied to produce first averaged data and averaging the first averaged data with 
other first averaged data or with an input data item to which a filter coefficient that is 
twice as large as the smallest filter coefficient is to be applied. 

3. A method according to claim 1 or 2, the sequence of averaging calculations 
further comprising successive further averagings, wherein averagings are performed 
between sets of input data, the sets of input data comprising averaged input data or 
individual input data items, averages only being performed between sets of input data 
where the sum of the filter coefficients to be applied to each set of input data are equal. 

4. A method of filtering data according to any one of claims 1 to 3, comprising 
repeatedly performing the method steps such that an array of m x n filter coefficients 
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are applied to a plurality of m input data items to produce a plurality of n output data 
items. 

5. A method according to claim 4, wherein the step of performing a sequence of 
averaging calculations includes selecting the input data that is to be averaged in 
dependence upon the frequency that two pieces of input data have common filter 
coefficients applied to them, and where possible selecting to average two pieces of 
data that have common filter coefficients applied to them a plurality of times, such that 
the calculated average of the input data can be re-used. 

6. A method according to any preceding claim, comprising the further step of, 
prior to applying the plurality of filter coefficients to the input data, arranging the 
coefficients such that they all have values that are a negative power of two and they 
sum to one. 

7. A method according to claim 6, wherein the step of arranging the filter 
coefficients such that each coefficient has a value that is a negative power of two. 
comprises splitting any filter coefficient that does not have this property into a sum of 
components, each component having a value that is a negative power of two. 

8. A method according to any preceding claim, wherein the filter coefficients 
relate to a low-pass filter. 

9. A method according to claim 8, wherein the filter coefficients relate to a 
deblocking filter. 

10. A method according to any preceding claim, wherein the input data relates to 
an array of pixels. 

11. A method according to claim 10, wherein the input data comprises an array of 
ten pixels, the filter coefficients being such that output data comprises an array of eight 
pixels. 
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12. A computer program product for applying a plurality of m filter coefficients to 
a plurality of m input data items comprising: 

filtering logic operable to control a processor to combine filter coefficients 
5 which each have a value that is a negative power of two and that sum to one, with a 
plurality of m input data items to produce a filtered output data item, by controlling the 
processor to perform a sequence of averaging calculations, the sequence of averaging 
calculations comprising: 

averaging input data items to which a smallest filter coefficient is to be applied 
1 0 to produce first averaged data; and 

averaging the first averaged data with other averaged input data or with input 
data items to which larger filter coefficients are to be applied the plurality of m filter 
coefficients being applied to the plurality of m input data items via a sequence of 
averaging calculations such that a data width of any calculated data does not exceed 
5 that of the input data items. 

13. A computer program product according to claim 12, wherein the sequence of 
averaging calculations comprises averaging two items of input data to which a smallest 
filter coefficient is to be applied to produce first averaged data and averaging the first 
3 averaged data with other first averaged data or with an input data item to which a filter 
coefficient that is twice as large as the smallest filter coefficient is to be applied. 

14. A computer program product according to claim 12 or 13, the sequence of 
averaging calculations further comprising successive further averagings, wherein 
averagings are performed between sets of input data, the sets of input data comprising 
averaged input data or individual input data items, averages only being performed 
between sets of input data where the sum of the filter coefficients to be applied to each 
set of input data are equal. 

15. A computer program product of filtering data according to any one of claims 
12 to 14, wherein the filtering logic is operable to control the processor to repeatedly 
perform the sequence of averaging calculations such that an array of m x n filter 
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coefficients are applied to a plurality of m input data items to produce a plurality of n 
output data items. 

1 6 A computer program product according to claim 1 5, wherein the filtering logic 
is further operable to control a processor to select pairs of input data items that are to 
be averaged in dependence upon the frequency that the pairs of input data items have 
common filter coefficients applied to them, such that where possible two input data 
items that have common filter coefficients applied to them a plurality of times are 
selected, such that a calculated average of the input data can be re-used. 



1 7. A computer program product according to any one of claims 12 to 16, further 
comprising: 

coefficient arranging logic arranged prior to the filtering logic and operable to 
control a processor to arrange the filter coefficients such that each coefficient has a 
1 5 value that is a negative power of two and the coefficients sum to one. 

18. A computer program product according to claim 17, wherein the coefficient 
arranging logic is operable to control a processor to split any filter coefficient that does 
not have a value that is a negative power of two into a sum of components, each 

20 component having a value that is a negative power of two. 

19. A computer program product according to any one of claims 1 2 to 1 8, wherein 
the filtering logic comprises a low-pass filter. 

25 20. A computer program product according to claim 1 9, wherein the low-pass filter 
comprises a deblocking filter. 
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21. A computer program product according to any one of claims 1 2 to 20, wherein 
the input data relates to an array of pixels. 
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22. A computer program product according to claim 21, wherein the input data 
comprises an array of ten pixels, the filter coefficients being such that filtered data 
comprises an array of eight pixels. 

23. A computer program product according to any one of claims 12 to 22, wherein 
the input data items each comprise 8 bits. 

24. A computer program product according to any one of claims 12 to 23, wherein 
the filtering logic comprises a plurality of SIMD instructions. 

25. A computer program product according to claim 24 when dependent on claim 
23, wherein the filtering logic is operable to process four input data items in parallel. 

26. A computer program product according to claim 24 or 25, wherein the filtering 
logic is operable to set the carry flag to indicate if the filtered output data is to be 
stored or discarded. 

27. A data processing apparatus comprising a processor operable to filter input 
data, the data processing apparatus comprising: 

filtering logic operable to control the processor to apply a plurality of m filter 
coefficients which each have a value that is a negative power of two and which sum to 
one, to a plurality of m input data items to produce a filtered output data item, by 
controlling the processor to perform a sequence of averaging calculations, the 
sequence of averaging calculations comprising: 

averaging input data items to which a smallest filter coefficient is to be applied 
to produce first averaged data; and 

averaging the first averaged data with other averaged input data or with input 
data items to which larger filter coefficients are to be applied the plurality of m filter 
coefficients being applied to the plurality of m input data items via a sequence of 
averaging calculations, such that a data width of any calculated data does not exceed 
that of the input data. 
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28. A data processing apparatus according to claim 27, wherein the sequence of 
averaging calculations comprises averaging two items of input data to which a smallest 
filter coefficient is to be applied to produce first averaged data and averaging the first 
averaged data with other first averaged data or with an input data item to which a filter 

5 coefficient that is twice as large as the smallest filter coefficient is to be applied. 

29. A data processing apparatus according to claim 27 or 28, the sequence of 
averaging calculations further comprising successive further averagings, wherein 
averagings are performed between sets of input data, the sets of input data comprising 

10 averaged input data or individual input data items, averages only being performed 
between sets of input data where the sum of the filter coefficients to be applied to each 
set of input data are equal. 

30. A data processing apparatus according to any one of claims 27 to 29, wherein 
the filtering logic is operable to control the processor to repeatedly perform the 
sequence of averaging calculations such that an array of m x n filter coefficients are 
applied to a plurality of m input data items to produce a plurality of n output data 
items. 

31. A data processing apparatus according to claim 30, wherein the filtering logic 
is further operable to control the processor to select the input data that is to be 
averaged in dependence upon the frequency that two pieces of input data have 
common filter coefficients applied to them, and where possible to select to average 
two pieces of data that have common filter coefficients applied to them a plurality of 
times, such that the calculated average of the input data can be re-used. 

32. A data processing apparatus according to any one of claims 27 to 31, further 
comprising coefficient arranging logic, the coefficient arranging logic being operable 
before the filtering logic to control the processor to arrange the filter coefficients such 
that each coefficient has a value that is a negative power of two and the coefficients 
sum to one. 
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33. A data processing apparatus according to claim 32, wherein the coefficient 
arranging logic is operable to control a processor to split any filter coefficient that does 
not have a value that is a negative power of two into a sum of components, each 
component having a value that is a negative power of two. 

34. A data processing apparatus according to any one of claims 27 to 33, wherein 
the filtering logic comprises a low-pass filter. 

35. A data processing apparatus according to any one of claims 27 to 34, wherein 
the input data relates to an array of pixels. 

36. A data processing apparatus according to claim 35, wherein the input data 
comprises an array of ten pixels, the filter coefficients being such that filtered data 
comprises an array of eight pixels. 

37. A data processing apparatus according to any one of claims 27 to 36, wherein 
the input data items each comprise 8 bits. 

38. A data processing apparatus according to any one of claims 27 to 37, wherein 
the filtering logic comprises a plurality of SIMD instructions. 

39. A data processing apparatus according to claim 38 when dependent on claim 
37, wherein the processor comprises 32 bit registers which are operable to process four 
input data items in parallel. 

40. A data processing apparatus according to claim 38 or 39, the processor further 
comprising at least one carry flag, said filtering logic being operable to set the carry 
flag to indicate if filtered output data is to be stored or discarded. 

41. A method of filtering data substantially as hereinbefore described with 
reference to Figures 2 to 4. 
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«• A data processing apparatus substantially as hereinbefore described with 
reference to Figures 2 to 4. 

43. A computer program product substantially as hereinbefore described with 
reference to Figures 2 to 4. 
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ABSTRACT 
DATA FILTERING 



A method, computer program product and data processing apparatus for 
filtering data, in particular for use in deblocking filters. The method comprising 
applying a plurality of m filter coefficients which each have a value which is a 
negative power of two and which sum to one, to a plurality of m input data items to 
produce a filtered output data item, by performing a sequence of averaging 
calculations comprising averaging input data items to which a smallest filter 
coefficient is to be applied to produce first averaged data and averaging the first 
averaged, data with other averaged input data or with input data items to which larger 
filter coefficients are to be applied the plurality of m filter coefficients being applied to 
the plurality of m input data items via a sequence of averaging calculations such that a 
data width of any calculated data does not exceed that of the input data being 
averaged. 



Figure 3 A 
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